Survey on Exploratory Software Testing

Uuriv testimine on testimismeetod, millele pani aluse Cem Kaner 1983. aastal. Käesoleva magistritöö eesmärk on analüüsida, millises ulatuses kasutatakse uurivat testimist Eesti ja Soome tarkvaraettevõtetes ning milline on nende ettevõtete kogemus uuriva testimise osas. Samuti on eesmärgiks otsida seoseid rollide, töökogemuse, asukoha, ettevõtte suuruse, ettevõtte ajaloo, testimise korralduse ja uuriva testimise kasutamise viisi vahel. Lisaks pakub huvi, mida testijad arvavad uuriva testimise meetodist, millised on uuriva testimise eelised ja puudused ning kuidas plaanitakse uurivat testimist täiustada. Eesmärgi saavutamiseks viidi läbi küsitlus isikute hulgas, kes on tööalaselt seotud testimisega. Kahe kuu jooksul saatsid vastused 61 küsitletut, kellest 50 on pärit Eestist ja Soomest. Töös tehakse kokkuvõte saadud vastustest ning vastatakse kolmele peamisele uurimisküsimusele, milleks on: Mis iseloomustab küsitlusele vastanuid, kes kohaldavad uurivat testimist Eestis ja Soomes ja milline kogemus on nende ettevõtetel uuriva testimise kasutamisega? Millised faktorid mõjutavad uuriva testimise kasutamist Eesti ja Soome ettevõtetes? Mida arvavad tarkvaratehnikud uurivast testimisest? Küsitlusele vastanute hulgas on kõige rohkem testijaid (48%) ja juhtivaid testijaid (36%). Ligi 50% vastanutest on rohkem kui 5 aastat töökogemust. Eesti testijatest kasutab uurivat testimist suurem osa (50%) kui Soome testijatest (38%). Uurivat testimist kohaldatakse väikestes ettevõtetes (vähem kui 50 töötajat, 56%) rohkem kui suurtes (32%). Üle 50% ettevõtetest, kus uurivat testimist kohaldatakse, on vanemad kui 5 aastat. Ligikaudu 76% vastanutest, kes kasutavad uurivat testimist, kuuluvad oma ettevõttes eraldiseisvasse testimisorganisatsiooni. Töökogemus tundub olevat Eestis ja Soomes peamine faktor, mis mõjutab, kas uurivat testimist kasutatakse või mitte. Vastanud, kellel on pikaajaline töökogemus, kasutavad uurivat testimist rohkem kui need, kellel on väiksem töökogemus. Ettevõtte suurus on märgatavalt erinev Eesti ja Soome uurivat testimist kohaldavate ettevõtete vahel. Eestis eelistavad kasutada uurivat testimist vastanud väikestest ettevõtetest, kuid Soomes kasutavad uurivat testimist rohkem suurtest ettevõtetest pärit vastajad. Vastajad toovad välja, et uuriv testimine soosib loovust ning on efektiivne. Samas arvavad mõned vastanud, et uuriv testimine nõuab testijalt suuremat testimisalast võimekust, mis tähendab, et uuriv testimine ei sobi igaühele. Mõned vastanud saavad kasutada uurivat testimist vaid vähestes olukordades, mistõttu leiavad nad, et uuriv testimine ei ole paindlik. Peamine probleem uuriva testimisega on, et seda on raske salvestada. Enamik vastanutest on valmis muutma uuriva testimise salvestamise strateegiat.Exploratory Testing (ET) is a testing method that was created by Cem Kaner in 1983 [1]. [Pleease double-check: In the references (Section 6) it says Kaner, Falk and Nguyen, and the year is 1993. ET was created in 1983, but this was mentioned in references [6] in 1993]The objective of this thesis is to investigate to what extent ET is currently applied in Estonian and Finnish software companies and what experience the companies have with using ET. Also, we want to know more about the relationship between role, working experience, location, company size, company history, type of test organization and the way how ET is used. In addition, we try to find out how testers think about ET, e.g., the advantages and disadvantages of ET, and how they plan to improve ET. To achieve the objective of this thesis, we decided to use a survey to gather replies from testing relevant roles. After over 2 months, we got 61 replies. 50 are from Estonia and Finland. We summarize all information and answer three main research questions of this study, which are: RQ 1: What characterizes those respondents that apply ET in Estonia and Finland and what experience do their companies have with using ET? RQ 2: Which factors have an influence on using ET in Estonian and Finnish companies? RQ 3: How do software engineers think about ET? Respondents in Estonian and Finnish companies who apply ET are mostly testers (48%) and test managers (36%). Close to 50% of the respondents have more than five years of working experiences. More Estonian testers (50%) use ET than Finnish testers (38%). ET has been applied in small companies (less than 50 employees, 56%) relatively more often than in large companies (32%). Over 50% of the companies applying ET are older than 5 years. About 76% of the respondents who use ET are members of a separate testing organization within their company. “Working experience” seems to be the main factor that influences whether a company applies ET in Estonia and Finland. Respondents who have much working experience use ET more than those who have less working experience.“Company size” is significantly different between those companies which apply ET in Estonia as compared to Finland. Respondents who are from smaller companies in Estonia prefer using ET, but in Finland, bigger companies’ respondents use ET more than those who are from smaller companies. Respondents consider that ET supports creativity, and ET are efficient and effective. Meanwhile some respondents think ET requires higher testing ability for testers, which means ET does not suit for everyone. Some respondents can use ET for so limit testing situations that make them considering that ET is inflexible. The main problem which ET brings is“hard to record”. Most respondents are willing to change the recording strategy of ET

Kano-sarnase mudeli kasutamine avatud innovatsiooni saavutamiseks nõuete analüüsi protsessis

Kui viiakse läbi nõuete analüüsi (inglise k Requirements Engineering, lühend RE), siis sageli järjestatakse nõuded nende olulisuse alusel (inglise k requirements prioritization), et saada selgust, milliste välja pakutud nõuetega funktsioon peaks tarkvaral olemas olema, seega sõltub tarkvara analüüsist tarkvara majandusliku väärtuse suurendamisega seotud otsuste tegemine. Tänapäeval arenevad tooted väga kiiresti ning ka nõuete olulisuse alusel järjestamine (inglise k requirements prioritization) on muutunud kiiremaks. Ettevõtted sooviksid saada kasutajatelt kiiret tagasisidet selle kohta, mis peaks olema järgmises mudelis olemas. Üks häid lahendusi sellele on Kano mudel (inglise k Kano model). Kano mudel selgitab välja kasutajate rahulolu ja toodete tunnuste vahelise suhte. See meetod liigitab kasutajate eelistused nende tähtsuse järjekorras, seega toetab see ka nõuete olulisuse järjekorra moodustamist. Aga Kano mudeli rakendamine on kallis ja aeganõudev ning seda ei saa kiiresti korrata. Veelgi enam – see mudel on keeruline väikeste ettevõtete jaoks, sest neil ei tarvitse olla piisavalt rahalisi jm vahendeid, et kasutajatega ühendust võtta ja neid intervjueerida. See omakorda paneb väikesed ettevõtted, eriti just idufirmad, ebavõrdsesse olukorda suurte ettevõtetega. Et sellele probleemile lahendust leida ja Kano mudeli kasutuselevõttu lihtsamaks ning odavamaks teha, arvame, et Kano mudelit tuleks arendada kahel viisil. Esiteks tuleks kasutada tasuta võrgus leiduvaid kirjalikke andmeid, mida saaks asendada intervjueeritavatelt kogutud vastustega. Teiseks – selleks, et hakkama saada võrgust kogutud kirjalike andmete suure mahuga, ning et kaasa aidata korrapärastele analüüsidele, peaks andmete analüüsimine olema automaatne. Selle uurimuse eesmärk on välja pakkuda meetodeid, et kasutajate avamusi, mis on võrgus saadavatest vabadest allikatest kogutud, (semi-)automaatselt liigitada, ja seda selleks, et aidata otsustajatel otsustada, millised tarkvara nõuded järgmises mudelis kindlasti olemas peaksid olema. Et seda uurimuse eesmärki saavutada, pakume me välja avatud innovatsiooni nõuete analüüsi (OIRE) meetodi, mille abil saavad tarkvarafirmad parema ülevaate kasutajate vajadustest ja sellest, kuivõrd rahul on nad olemasolevate toodetega.When Requirements Engineering (RE) is applied, requirements analysis is often used to determine which candidate requirements of a feature should be included in a software release. This plays a crucial role in the decisions made to increase the economic value of software. Nowadays, products evolve fast, and the process of requirements prioritization is becoming shorter as well. Companies benefit from receiving quick feedback from end users about what should be included in subsequent releases. One effective approach supporting requirements prioritization is the Kano model. The Kano model defines the relationship between user satisfaction and product features. It is a method used to classify user preferences according to their importance, and in doing so, supports requirements prioritization. However, implementing the Kano model is costly and time-consuming, and the application of the Kano model cannot be repeated quickly. Moreover, this is even more difficult for small companies because they might not have sufficient funds and resources to contact end users and conduct interviews. This puts small businesses, especially start-ups, at an unfair disadvantage in competing with big companies. To address this problem and make the application of the Kano model simpler, faster, and cheaper, we propose evolving the Kano model in two aspects. First, free online text data should be used to replace responses collected from interviewees. Second, in order to handle the higher amount of data that can be collected from free online text data and in order to facilitate frequent analyses, the data analysis process should be automated. The goal of this research is to propose methods for (semi-)automatically classifying user opinions collected from online open sources (e.g., from online reviews) to help decision-makers decide which software requirements to include in subsequent product versions. To achieve this research goal, we propose the Open Innovation in Requirements Engineering (OIRE) method to help software organizations gain a better understanding of user needs and satisfaction with existing products. A key element of the OIRE method is its Kano-like model. This Kano-like model mimics the traditional Kano model, except that it uses data from online reviews instead of interviews conducted with select focus groups.https://www.ester.ee/record=b527385

How is Exploratory Testing Used? : A State-of-the-Practice Survey

Jufo_ID: 59031 ; lyhenne ESEM.Context: Exploratory Testing has experienced a rise in popularity in the industry with the emergence of agile development practices, yet it remains unclear, in which domains and how it is used in practice. Goal: To study how software engineers understand and apply the principles of exploratory testing, as well as the specific advantages and difficulties they experience. Method: We conducted an online survey in the period June to August 2013 among Estonian and Finnish software developers and testers. Results: Our main findings are that the majority of testers, developers, and test managers using ET, (1) apply ET to usability- critical, performance-critical, security-critical and safety-critical software to a high degree; (2) use ET very flexibly in all types of test levels, activities, and phases; (3) perceive ET as an approach that supports creativity during testing and that is effective and efficient; and (4) find that ET is not easy to use and has little tool support. Conclusions: The high degree of application of ET in critical domains is particularly interesting and indicates a need for future research to obtain a better understanding of the effects of ET in these domains. In addition, our findings suggest that more support to ET users should be given (guidance and tools).Peer reviewe